1. Field of the Invention
The present invention relates to the present invention pointing position detection device, presentation system, method, and an information storage medium.
2. Background Art
In case a presenter is assumed to be giving a presentation while viewing a screen-image that appears on a screen, the presenter goes through the presentation while using a pointing device to indicate any of various points on the screen.
With such a system, it is important that the presenter can indicate any point on the screen in a simple manner, and it is also important that the system can reliably perceive that pointing position and can copy the pointing position for image processing.
However, when the presenter indicates desired points in a prior-art system, a problem arises in that a special pointing device has to be used therefor, which is extremely inconvenient for the user.
If, for example, a laser pointer is used to indicate a desired point on the screen, the pointing position is difficult to see if the surroundings of the screen are not dark enough. Since the pointing position is displayed as a small dot, this also causes a problem in that the pointing position is difficult to see.
If a pointing device such as a mouse is used for giving a presentation while moving a cursor position that is displayed on the screen, the presenter must interrupt the presentation to a certain degree, to operate the mouse. This causes problems in that the presentation cannot proceed smoothly and the operation is extremely inconvenient.
An objective of the present invention is to provide a pointing position detection device, presentation system, and method, and a computer-readable medium in which a pointing position within a display region can be sensed automatically and accurately, without using a special device as a point indication tool.
In order to achieve the objective described above, a pointing position detection device of the present invention comprises:
image-acquisition means for acquiring an image of a display region in which an image is displayed; and
position detection means for detecting a pointing position from an image region of a pointing image included within the display region, based on an image-acquisition signal of the image-acquisition means.
A method in accordance with the present invention comprises:
an image-acquisition step of acquiring a picture of a display region in which an image is displayed; and
a position detection step of detecting a pointing position from an image region of a pointing image included within the display region.
When the presenter indicates a desired position in the display region in which a image is displayed with a predetermined indication tool such as a rod or finger, this aspect of the invention ensures that this indicated position can be detected as the pointing position, based on the image region of the pointing image generated by this indicator.
This makes it possible to implement a pointing position detection device that is extremely convenient to use.
A pointing position detection device in another aspect of the present invention comprises:
image-acquisition means for acquiring an image of a display region in which an image is displayed;
reference-value setting means for setting a reference value for separating an image region of a pointing image included within the display region, from an image-acquisition signal of the image-acquisition means; and
position detection means for detecting a pointing position from the image region of the pointing image, by comparing the image-acquisition signal and the reference value and thus extracting the image region of the pointing image,
wherein the reference-value setting means is configured to be capable of modifying the reference value.
A pointing position detection method in this aspect of the present invention comprises:
an image-acquisition step of acquiring an image of a display region in which an image is displayed; and
a position detection step of comparing an image-acquisition signal with a reference value for separating an image region of a pointing image and thus extracting the image region of the pointing image, then detecting a pointing position from the image region of the pointing image,
wherein the position detection step comprises a step of modifying the reference value for separating the image region of the pointing image included within the display region from the image-acquisition signal of an image-acquisition means.
This aspect of the present invention extracts the image region of a pointing image created by the indicating action of an operator within a display region in which an image is displayed, from an image acquired by the image-acquisition means, and thus automatically detects a pointing position.
When the image-acquisition signal that is output from the image-acquisition means is that of a monochrome image, for example, a binary threshold value for extracting the image region of the pointing image from this image-acquisition signal could be set as the reference value.
This reference value is basically set to a value that is lower than the black-level brightness of an image acquired by the image-acquisition means. This is because the color in the screen image that has the lowest brightness is black, so the brightness of the image region of the pointing image is lower than the black brightness level.
If the margin between the black level of the image displayed on the display region and the reference value for the binary-encoding is too large in this case, a problem will arise in that the brightness level of the image region of the pointing image will not fall as far as below that reference level and it will not be possible to detect the image region of the pointing image.
Conversely, when the margin for the reference value is too small, the image target that is not the image region of the pointing image will be recognized erroneously as the image region of the pointing image. When the brightness within the display region varies somewhat due to ambient environmental conditions, for example, this will cause a problem in that the variation will be perceived erroneously as the image region of the pointing image.
Therefore, if this reference value is not set to a value that is suitable for the environment in which the system is used, a problem will arise in that it will be difficult to detect the image region of the pointing image appropriately.
Another aspect of the present invention is configured to make it possible to modify the reference value that is used for separating the image region of a pointing image from an image-acquisition signal. This solves the problem described above and makes it possible to accurately extract the image region of a pointing image that is formed on the display region, and thus detect the pointing position automatically.
The position detection means or step in this aspect may be configured in such a manner that a predetermined position of a shadow region of the pointing image included within the display region is detected as the pointing position.
Alternatively, the position detection means or step may be configured in such a manner that a predetermined position of a real-image region of the pointing image included within the display region is detected as the pointing position.
Alternatively, the position detection means or step may be configured in such a manner that the pointing position is detected from the image region of the pointing image having a rod-like shape included in the display region.
The position detection means or step may also be configured in such a manner to capture a rod-shaped image region as the image region of the pointing image, based on the continuous shape of the image region including within the display region, and detect a tip position of the image region as the pointing position.
A presentation system of the present invention comprises:
image display means for displaying an image;
image-acquisition means for acquiring an image of a display region in which an image is displayed;
position detection means for detecting a pointing position from an image region of a pointing image included within the display region based on an image-acquisition signal of the image-acquisition means; and
cursor control means for controlling a position of a cursor included within the image displayed by the image display means, based on the detected pointing position.
A computer useable information embodied on computer-readable medium in this aspect of the present invention comprises:
information for detecting a pointing position of an image region of a pointing image included in a display region, based on an image-acquisition signal of the display region; and
information for controlling a position of a cursor included within an image displayed by an image display means, based on the detected pointing position.
A method in another aspect of the present invention comprises:
an image display step of displaying an image;
an image-acquisition step of acquiring an image of a display region in which an image is displayed;
a position detection step of detecting a pointing position of an image region of a pointing image included within the display region from an image-acquisition signal of the image-acquisition means; and
a cursor control step of controlling a position of a cursor included within the image displayed by the image display means, based on the detected pointing position.
With this aspect of the invention, a presenter can point at any desired position on the display region with a predetermined indication tool, such as a rod-shaped indication tool or a finger, and that pointing position will be detected and the position of a cursor will be controlled. This ensures that the presentation can proceed smoothly.
Note that the indication tool that creates the shadow or actual image of the pointing image is preferably formed to have a slightly larger pointing position at the tip of the rod-shaped indication tool, and a non-reflective material is used to make this pointing position. This ensures that the tip portion of the indicator rod within the acquired image can form a shadow or actual image reliably, and thus the pointing position can be detected more accurately.
The presentation system in accordance with this aspect of the invention controls the position of a cursor displayed on the display region, based on the detected pointing position. This ensures that the cursor moves to follow the position indicated by the presenter so that the pointing position can be displayed in an extremely easy-to-see manner, not only for the presenter but also for the audience watching the presentation. As a result, the presentation can proceed in a form that is natural to both the presenter and the audience, making it possible to implement a presentation system that is convenient to use and also easy to understand.
The above described position detection means, information, or step could be configured in such a manner that a predetermined position of a shadow region of the pointing image included within the display region is detected as a pointing position.
This makes it possible for any pointing position indicated by the presenter within the display region to be detected as a shadow region on the display region.
Alternatively, this position detection means, information, or step could be configured in such a manner that a predetermined position of a real-image region of the pointing image included within the display region is detected as a pointing position.
This makes it possible for any pointing position indicated by the presenter within the display region to be detected as a real-image region on the display region.
Thus when the presenter indicates a certain position with an indication tool such as a rod, this aspect of the invention makes it possible for that pointing position to be detected as a shadow region or real-image region of the indication tool on the display region.
This position detection means, information, or step is preferably configured in such a manner that a rod-shaped image region included within the display region is captured as the image region of the pointing image, and a tip position of the image region is detected as the pointing position.
This ensures that detection targets other than the rod-shaped image region are determined to be noise within all the possible detection targets and thus are not perceived to be the pointing image. This enables accurate pointing position detection that is not affected by factors such as noise and environment conditions.
In addition, the tip of the image region of the rod-shaped pointing image is detected as the pointing position, so that the presentation can proceed with a sensation as if the user is giving the presentation while using a finger or rod to indicate as if with a black plate or the like.
This position detection means, information, or step is also preferably configured in such a manner that a rod-shaped image region is captured as the image region of the pointing image, based on the continuous shape of the image region included within the display region.
This configuration ensures that the pointing image can be reliably separated from noise images comprises within the acquired image, making it possible to detect the pointing position even more reliably.
The cursor control means, information, or step in this aspect is preferably configured in such a manner that a plurality of cursors is displayed simultaneously within the display region and also each cursor position is controlled independently.
Another aspect of the cursor control means, information, or step comprises means, information, or a step for fixedly displaying the cursor at a desired position within the display region.
This configuration ensures that a cursor can be made to move to follow the position indicated by the presenter, so that the audience can perceive the position indicated by the presenter as the position of the cursor, in an easily understandable fashion.
In a still further aspect of the present invention, cursor control is provided for a plurality of pointing images, as described in the example below. Assume, for example, that the pointing position is to be displayed individually for each of the presenter and a participant. When a cursor at the position of the pointing image indicated by the participant (assistant) can be controlled in addition to control over the cursor position indicated by the presenter, they can both see at a glance a location that requires explanation or a location of a question. This makes it possible to implement natural discussions and conferences, enabling highly understandable presentations.
Similarly, when the presenter wishes to compare two locations on the displayed screen-image, the presenter can set a first cursor at the position of a first explanatory point by pointing at it directly, then set the cursor to be fixed at that position and point directly at the location of the comparison point. This enables the audience to see the two marker locations simultaneously, so that the audience can simply search for the two marker locations.
It is also possible to hold the cursor position then return to the presentation smoothly, when the presenter is searching through reference material and thus is separated from the display region and cannot indicate it. This makes it possible to efficiently implement a presentation, without causing the attention of the participants to wander.
A further aspect of this cursor control means, information, or step preferably controls a cursor position to indicate the detected pointing position, at a position having no overlap with the image region of the pointing image.
Thus the position at which the cursor is displayed is located such that it does not overlap the position indicated by the presenter on the display region, so that the pointing position can be conveyed to the audience watching the presentation in an easy-to-understand and accurate fashion.
The above described cursor control means, information, or step may control a cursor size to be displayed according to a tip size of an image region of the pointing image having a rod like shape.
For example, when the tip of the pointing image covering region is small, the pointing position or the cursor can be made easier for the audience to see, by displaying the cursor larger.
A device or system in this aspect of the present invention may comprise:
a screen having the display region,
wherein the image display means comprises an image projection means for projecting an image onto the display region; and
optical means of the image-acquisition means controls image-acquisition conditions in linkage with optical means of the image projection means.
In a computer useable information embodied on computer-readable medium as another aspect of the present invention, information for controlling optical means of the image-acquisition means preferably comprises information for controlling image-acquisition conditions in linkage with optical means forming an image projection means for projecting an image onto the display region.
Ordinarily, the presenter adjusts the focus and zoom of the optical means, such as a lens, of the image projection means while watching the screen-image displayed on the display region. On the other hand, optical adjustment of the image-acquisition means is often done while the presenter is watching the acquired image. With this aspect of the invention, the presenter is released from making these two troublesome adjustment operations by control that links the optical adjustment of the image-acquisition means to the optical adjustment of the image projection means, thus making the system more convenient.
A device or presentation system in this aspect of the present invention preferably comprises:
a display having the display region,
wherein the image projection means is disposed in such a manner as to project an image from in front of or behind the screen.
A front-projection type of presentation system, which projects an image from the front of a screen that is a display region, makes it possible for the presentation to preceded while a large screen image for the presentation is projected onto the screen from the image display means. However, in this case, it is often difficult to touch the screen image directly, or point to a position thereon. In this case too, this aspect of the invention makes it possible for the presenter, who is positioned between the image-acquisition means and the screen, to cause a desired position to be perceived as a pointing position, by creating a shadow on the screen in such a manner as to interrupt part of the light path of the projection means, or by creating an actual image of the indication means within the acquired image in such a manner as to interrupt part of the light path of the image-acquisition means.
Similarly, in a rear-projection type of presentation system, which projects an image from the rear of the display region, the presenter can cause a shadow region or real-image region of a pointing image to appear at a position on the display region indicated by the presenter. Therefore the pointing position can be detected from the shadow or actual image thereof.
A still further aspect of the presentation system in accordance with the present invention comprises:
image display means for displaying an image;
image-acquisition meanas for acquiring an image of a display region in which an image is displayed;
reference-value setting means for setting a reference value for separating an image region of a pointing image included within the display region from an image-acquisition signal of the image-acquisition means;
position detection means for detecting a pointing position from the image region of the pointing image, by comparing the image-acquisition signal and the reference value and thus extracting the image region of the pointing image; and
cursor control means for controlling a position of a cursor included within the image displayed by the image display means, based on the detected pointing position,
wherein the reference-value setting means is configured to be capable of modifying the reference value.
A computer useable information embodied on computer-readable medium for controlling a presentation system, as a still further aspect of the present invention, comprising:
reference-value setting information for setting a reference value for separating an image region of a pointing image included within a display region from an image-acquisition signal of image-acquisition means;
position detection information for detecting a pointing position from the image region of the pointing image, by comparing the image-acquisition signal and the reference value and thus extracting the image region of the pointing image; and
cursor control information for controlling a position of a cursor included within an image displayed by image display means, based on the detected pointing position,
wherein the reference-value setting information comprises information for modifying the reference value.
As a still further aspect of the present invention, a method of detecting a pointing position for a presentation comprises:
an image-acquisition step of acquiring an image of a display region in which an image is displayed; and
a position detection step of comparing an image-acquisition signal with a reference value for separating an image region of a pointing image and thus extracting the image region of the pointing image, then detecting a pointing position from the image region of the pointing image; and
a cursor control step of controlling a position of a cursor included within an image displayed by an image display means, based on the detected pointing position,
wherein the position detection step comprises a step of modifying the reference value for separating the image region of the pointing image included within the display region from the image-acquisition signal of an image-acquisition means.
This aspect of the invention makes it possible to modify the reference value that is used for separating the image region of a pointing image from an image-acquisition signal. This ensures that the image region of a pointing image that is formed on the display region can be extracted accurately and, as a result, the pointing position can be detected automatically and the position of the cursor can be controlled easily, enabling the presentation to proceed smoothly.
The position detection means, information, or step may be configured in such a manner that a predetermined position of a shadow region of the pointing image included within the display region is detected as the pointing position.
Alternatively, this position detection means, information, or step may be configured in such a manner that a predetermined position of a real-image region of the pointing image included within the display region is detected as the pointing position.
The position detection means, information, or step is preferably configured in such a manner that a rod-shaped image region included within the display region is captured as the image region of the pointing image, and a tip position of the pointing image is detected as the pointing position.
The position detection means, information, or step also preferably captures a rod-shaped image region as the image region of the pointing image, based on the continuous shape of the image region included within the display region.
The device and system in this aspect of the present invention preferably further comprises:
calibration pattern image generation means for displaying a predetermined calibration pattern image from the image display means to the display region,
wherein the reference-value setting means automatically sets a reference value for extracting the image region of the pointing image, based on data of the calibration pattern image and the image-acquisition signal.
A computer useable information embodied on computer-readable medium in this aspect of the present invention preferably comprises calibration pattern control information for displaying a predetermined calibration pattern image from the image display means to the display region,
wherein the reference-value setting information comprises information for automatically setting a reference value for extracting an image region of the pointing image, based on data of the calibration pattern image and the image-acquisition signal.
More specifically, this calibration pattern image is projected from the image display means towards the display region when the image display means starts operating, or during a mid-operation update.
In this case this time, the reference-value setting means automatically sets the reference value that is used for extracting the image region of the pointing image from the acquired image, based on an image-acquisition signal obtained from the acquisition of an image of the displayed calibration pattern and brightness data for this calibration pattern image. If the image-acquisition signal is a monochrome image signal, for example, the reference value is set automatically to have an optimal margin with respect to the brightness level of black.
In this case, the device and system in this aspect preferably further comprises:
indication means for setting a reference value,
wherein when a reference-value setting instruction is input by the indication means:
the calibration pattern image generation means causes the display of a calibration pattern image from the image display means on the display region; and
the reference-value setting means automatically sets a reference value for extracting the image region of the pointing image, based on data of the calibration pattern image and the image-acquisition signal.
Even with a presentation system that is used in the same location, the optimal reference value often varies according to factors such as the time at which it is used and the ambient environment. For example, the optimal reference value will differ when the system is used during the day from when it is used at night. Even when the system is used at the same time of day, this optimal reference value will change depending on whether the day is sunny or cloudy.
In addition, if the presentation extends over a long period of time, it can often happen that the ambient brightness will change between the start of the presentation and a point at which a certain amount of time has elapsed, so that the optimal reference value will also change.
In such a case too, this aspect of the invention could be configured in such a manner that a calibration pattern image is displayed onto the display region from the image display means, by using the indication means to specify that the reference value is reset. This ensures that the reference-value setting means automatically resets the optimal reference value on the basis of the image-acquisition signal for this calibration pattern and calibration pattern data.
The reference-value setting means preferably divides the display region into a plurality of regions and sets an individual reference value for each of the divided regions.
In such a case, the calibration pattern image generation means is preferably configured in such a fashion that a predetermined calibration pattern image having a calibration pattern for each of these partial areas is displayed on the display region.
In another aspect of the invention, this display region is preferably divided into at least a corner area that can easily become comparatively dark and a center area that can easily become relatively bright, and an individual reference value is set for each of these partial areas.
In a still further aspect of the computer useable information embodied on computer-readable medium of the present invention, the reference-value setting information preferably comprise information for dividing the display region into a plurality of regions and automatically setting an individual reference value for each of the divided regions.
When a projector is used as the image display means and the display region is curved or the brightness of the black level of the screen image becomes uneven due to variations in the brightness of the projected image, it will be necessary set an optimal reference value to suit each area.
With this aspect of the invention, the image region of the pointing image can be extracted reliably from the acquired image and the pointing position can be detected, even if there are variations in brightness over the display region, by dividing the display region into a plurality of regions and setting an individual reference value for each partial area.
The calibration pattern image generation means preferably displays a predetermined calibration pattern image formed by combining of a plurality of regions having different brightnesses on the display region.
In other words, since it often happens that the calibration pattern displayed from the image display means during the reference value setting process sets black, with the lowest brightness level, as the reference, for example, the entire area of the calibration pattern image could be set to a completely black level. However, when the same black is displayed as an image on the display region, the brightness level obtained by the image-acquisition means when it acquires that black will be slightly different if the surroundings are completely black and if there are areas other than black in the surroundings.
With this aspect, the configuration is such that a calibration pattern image formed of a combination of a plurality of regions having different brightness is displayed and the reference value can be set from this calibration pattern image, in such a manner that the reference value can be set reliably and also to the optimum value even in the above cases. This makes it possible to extract the image region of the pointing image more accurately from the image-acquisition signal, and thus detect the pointing position.
In particular, a suitable value can be set for the reference value for each of the divided region, by a configuration in which a combined pattern of a plurality of regions having different brightnesses is displayed in the display region, as a calibration pattern image set for each divided region.
Yet another aspect of the presentation system of the present invention comprises:
image display means for displaying an image;
image-acquisition means for acquiring an image of the display region; and
processing means for detecting a pointing position, based on an image-acquisition signal of the image-acquisition means.
A method of detecting a pointing position for a presentation in a further aspect of the present invention comprises:
an image-acquisition step of acquiring an image of a display region in which an image is displayed; and
a step of detecting a pointing position from an image region of a pointing image, based on an image-acquisition signal.
In this case, the processing means or position detection step may be configured in such a manner that the pointing position is detected from a shadow region of the pointing image of the presenter included within the display region.
Alternatively, this processing means or position detection step may be configured in such a manner that the pointing position is detected from a real-image region of the pointing image of the presenter included within the display region.
When the presenter indicates a certain position with a finger or a rod-like indication tool in this aspect of the invention, that pointing position is detected as a region of the shadow or the actual image of the finger or indication tool on the display region, and the pointing position is copied for data processing.
This makes it possible to implement a system that enables a presentation which feels more human, in a form that is particularly natural to both the presenter and the audience.
The processing means or position detection step in this aspect preferably comprises a pointing position detection means or step for detecting the pointing position of the presenter from the region of the rod-shaped pointing image included within the display region.
The processing means or position detection step is preferably further configured to enable the detection of a plurality of the pointing positions included within the display region.
In an even further aspect of the presentation system in the present invention, it is preferable that the image display means comprises an image projection means for projecting an image on a screen, and is disposed in such a manner as to project an image from in front of or behind the screen.
Optical means of the image-acquisition means is preferably configured to enable control of image-acquisition conditions, linked to optical means of the image projection means.
The image-acquisition means is preferably configured in such a manner that the image-acquisition area is freely adjustable with respect to the display region.
This makes it possible to implement a system that is extremely convenient to use, because the image-acquisition area can be seen as required to suit the environment in which it is used.
In this case, it is more preferably to configure the image-acquisition means and the image display means as an integral unit, to increase the convenience of the system.
This image-acquisition means is preferably configured in such a fashion that the image-acquisition direction thereof can be adjusted vertically.
This configuration makes it possible to prevent any distortion of the acquired image caused by the environment in which the image display means and image-acquisition means are used, thus making it possible to detect the pointing position more accurately from the acquired image.
The processing means or position detection step preferably comprises:
cursor control means or a cursor control step, for controlling a cursor position included within the image displayed by the image display means, based on the detected pointing position.
The cursor control means or step is further preferably to display a plurality of cursors simultaneously within the display region and control each cursor position independently.
This cursor control means or step is further preferable to comprise a position-fixing means or step for fixedly displaying the cursor at a desired position within the display region.
The system preferably comprises a polarizing plate which is provided in a light path of the image-acquisition means and absorbs light projected from the image projection means.
In this case, another polarized plate is preferably provided in the light path of the image projection means as well, where this polarized plate has a direction of polarization that differs from that of the polarized plate provided in the light path of the image-acquisition means.
This configuration ensures that the pointing position can be detected with comparatively little adverse effect from the light projected from the image projection means.
The presentation system in this aspect of the present invention preferably comprises illumination means for lighting a low-brightness level region of the display region.
In this case, the illumination means is preferably configured so that the direction of lighting thereof is freely adjustable with respect to the display region, in such a manner that any area having variations in the brightness of the display image caused by factors such as ambient lighting or environmental conditions, such as large corner areas that are affected by shading, can be selectively lit.
This ensures that the effects of shading on the screen can be reduced, making it possible to detect the pointing position more accurately.
The presentation system in this aspect of the present invention could further comprise:
function input means to which a predetermined function is allocated; and
means for performing data processing according to a function being input.
This ensures that the presentation can proceed while the presenter uses a mouse to perform an operation that is similar to a screen-image operation, by allocating a function to the left-click or right-click operation of a mouse. For example, a function such as dragging, underlining, or handwriting input could be allocated, so that the presentation can be given while using that function.
In addition, it is possible to use the cursor control means to indicate a plurality of points or enable a detailed discussion in a simple manner, by allocating an instruction for fixing the display of the position at which the cursor is displayed on the display region and another instruction for returning from the fixed display, thus increasing the effectiveness of the presentation.
In this case, when the presenter uses a rod-shaped indication tool for pointing, the function input means could be configured in such a manner that it is formed either integrally with that indication tool or separately from that indication tool, so that the presenter can operate it while holding it in a hand or wearing it on the body.